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ABSTRACT 


NASA is currently exploring strategies which can contribute to increased efficiency in 
performance of their various activities and is undertaking to evaluate the potential offered 
by a multitude of techniques in the Total Quality Management area. One such technique that 
has received considerable attention within the manufacturing community over the last few 
years is Taguchi design of experiments and optimization of design parameters. 

Taguchi methods are concerned with the quality design of an end-product and give specific 
focus to efficiency in the design process. The underlying philosophy of the Taguchi 
approach is that through introducing a planned structure to the experimentation carried 
out in the formative stages of design that a great deal of information regarding product or 
system performance can be gleaned in less time and at less cost than through traditional 
methods. 

The chief goals of the summer project have been twofold — first, for my host group and 
myself to learn as much of the working details of Taguchi analysis as possible in the time 
allotted, and, secondly, to apply the methodology to a design problem with the intention of 
establishing a preliminary set of near-optimal (in the sense of producing a desired 
response) design parameter values from among a large number of candidate factor 
combinations. 

The selected problem is concerned with determining design factor settings for an automated 
approach program which is to have the capability of guiding the Shuttle into the docking 
port of the Space Station under controlled conditions so as to meet and/or optimize certain 
target criteria. The candidate design parameters under study were glide path (i.e., 
approach) angle, path intercept and approach gains, and minimum impulse bit mode (a 
parameter which defines how Shuttle jets shall be fired). Several performance criteria 
were of concern: terminal relative velocity at the instant the two spacecraft are mated; 
docking offset; number of shuttle jet firings in certain specified directions (of interest 
due to possible plume impingement on the Station's solar arrays), and total RCS (a 
measure of the energy expended in performing the approach/docking maneuver). In the 
material discussed here, we have focused on a single performance criterion — total RCS. 
An analysis of the possibility of employing a multiobjective function composed of a 
weighted sum of the various individual criteria has been undertaken, but is, at this 
writing, incomplete. 

Results from the Taguchi statistical analysis indicate that only three of the original four 
posited factors are significant in affecting RCS response. A comparison of model simulation 
output (via Monte Carlo) with predictions based on estimated factor effects inferred 
through the Taguchi experiment array data suggested acceptable or close agreement 
between the two except at the predicted optimum point, where a difference outside a 
rule-of-thumb bound was observed. We have concluded that there is most likely an 
interaction effect not provided for in the original orthogonal array selected as the basis for 
our experimental design. However, we feel that the data indicates that this interaction is a 
mild one and that inclusion of its effect will not alter the location of the optimum. 
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INTRODUCTION 


A critical stage in the performance of the Shuttle's rendezvous and docking maneuvers with 
the Space Station will be the final approach, initiated at a separation of a few hundred feet 
between two spacecraft. This procedure can be supported, either entirely or in large 
degree, by an automated approach program designed to establish the approach path and 
control the approach velocity and attitude of the Shuttle relative to the Station, thus 
guiding the Shuttle into the Station's docking port. 

A central goal in the assessment of approach design options is that of determining a 
superior combination of design parameter settings so as to permit an approach/docking 
maneuver which, when evaluated against several quantifiable criteria, is projected to 
produce a near-optimal result. The set of design factors whose optimal levels are to be 
determined include the glide path approach angle ( 0 ), the minimum impulse bit setting 
(MIB) which defines the mode of Shuttle jet firings, and two gains — the path intercept 
gain (Kg r ) and approach gain factors (K x ) — which describe target velocity modification 

profiles for the approach. The set of performance indicators includes statistics indicative 
of the terminal conditions at the instant docking is to be achieved (e.g., the terminal 
velocity and docking offset); fuel expended in the approach (total RCS); and the number of 
Shuttle firings in specified directions, this last being of concern due to the possibility of 
plume impingement on the Station solar arrays. 

In the summer assignment, we have focused on a single performance criterion, total RCS, 
which we seek to minimize. Our evaluation medium was a simulation program developed 
within the Guidance & Prox Ops Section which models the dynamics of the problem and 
produces estimates for the various performance indicators, all of which are, of course, 
functions of the design parameter values set by the analyst. The methodology employed in 
searching for a near-optimum combination of these design factor values was Taguchi 
parameter optimization, a statistical design of experiments technique, the details of whose 
analytical steps are given in the sections below. 

EXPERIMENTAL DESIGN 

The domain for the factor settings in this preliminary study were as follows: Glide Path 
Angle - 0= -10°, 0= 0°, 0= +10°; Minimum Impulse Bit mode - MIB= norm z, MIB= low 
z; Path Intercept Gain - Kg r = 0.8T, Kg r = 1.0T, Ka r = 1.2T; Approach Gain - K x = 0.8T, 
K x = 1.0T, K x = 1.2T, where T for these last two factors is a theoretically derived value 
posited for each. Thus, our feasible region of interest is defined by four discrete decision 
variables with three 3-level factors and one 2-level factor. 

In an exhaustive search for the optimum set of design factor values, an evaluation of all 54 
setting combinations would be performed through a full-factorial experimental design. A 
by-product of the statistical analysis which follows compilation of the experiment data is 
a mathematical model which estimates all main factor effects and the effects of all 
interactions among these factors on the response variable (here, total RCS). Though 
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derivation of such a detailed model produces considerable insight into the phenomenon 
being modeled, there are also obvious disadvantages in this approach. .The time and expense 
involved in running experiments is frequently such that a large number of cases cannot be 
accommodated. 

Fortunately, in the majority of instances, it not necessary to perform a complete 
enumerative analysis. It has been observed that in engineering design problems the effects 
of the higher-order interaction terms — certainly those involving three or more factors 
— are very often insignificant and that, in many cases, most of the second-order terms 
have little effect on the response function. As a result, meaningful information may be 
gleaned through a considerably smaller battery of experiments, if the experimental design 
is constructed in a particular manner. These points are, in fact, the premises underlying 
Taguchi design of experiments. 

It not being evident that there would be any significant interactions among the factors, it 
was decided that we would employ the simplest experimental design that could accommodate 
all four factors. This was the Taguchi L g orthogonal array 1 , each of whose four columns 

could carry a 3-level factor. Note that, as pointed out earlier, one of the design 
parameters (MIB) was a 2-level factor, the remaining three parameters being 3-level 
factors. This mismatch between the factor levels of the problem and those provided for in 
the Lg array, is only apparent. In actual fact, using a dummy level technique, we simply 

choose a level designation to repeat as a third level value for the 2-level factor — for 
example, MIB 1= norm z, MIB 2 = norm z, MIB 3 = low z. The net result is that the estimate 

for the mean effect at one level (MIB= norm z) has twice the precision as that at the other 
(MIB= low z). In addition, the orthogonality of the experiment array is preserved. The L g 

layout for our matrix of experiments is given in Table 1 below. 


TABLE 1 : Lg MATRIX EXPERIMENT LAYOUT 

Experiment Number Eacloia Response 



£L 

MIB 

% 

K* 


1 

-1 0° 

norm z 

0.8T 

0.8T 

RCSi 

2 

-10° 

norm z 

1.0T 

1.0T 

rcs 2 

3 

-10° 

low z 

1.2T 

1.2T 

rcs 3 

4 

0° 

norm z 

1.0T 

1.2T 

rcs 4 

5 

0° 

norm z 

1.2T 

0.8T 

rcs 5 

6 

0° 

low z 

0.8T 

1.0T 

rcs 6 

7 

+ 1 0° 

norm z 

1.2T 

1.0T 

rcs 7 

8 

+ 10° 

norm z 

0.8T 

1.2T 

rcs 8 

9 

+ 10° 

low z 

1.0T 

0.8T 

RCSg 


1 A matrix experiment is termed an orthogonal array if the contrasts corresponding to all 
its columns are pairwise orthogonal (i.e., if the inner product of any two vectors of 
contrast weightings is zero). See Chapter 3 of Hicks [1] and Appendix A of Phadke [2J. 
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EXPERIMENT RESULTS 


Recall that we wished to evaluate the factor contributions with an eye towards minimizing 
the total RCS response — i.e., we were operating from a "smaller the better" basis, as 
termed in the Taguchi literature. For each of the nine response observations, the squared 
deviation from zero (the "ideal" target in the minimization of a nonzero objective) was 
computed in accordance with Taguchi’s concept of a quadratic quality loss function (see 
Chapter 2 of Roy [3]). The individual squared deviations were then transformed into dB 
readings 2 as follows: R j = -10 log 10 (RCSj). In the transformed domain we then wish to 
maximize the "signal" response. 

An analysis of variance was carried out on the transformed data with the result that all 
factors were statistically significant, with the exception of Ka r . Figure 1 below displays 

the estimated factor level effects in the original domain so that we see actual RCS projected 
effects. The optimum factor settings which are predicted to minimize total RCS are 0 = 
-10°, MIB = norm z and K x = 1.2T. 



Figure 1 : Mean Factor Effects on Total RCS 


2 Transformation of data into decibels is standard procedure in Taguchi analysis and offers 
an advantage in prediction, particularly when observed values are close to zero but cannot 
be negative. The transformation concept is also useful in assessing signal-to-noise ratio. 
See Phadke, Chapter 5. 
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The sum of squares estimated for Kg r was pooled with that for the ANOVA error sum of 

squares to produce an error mean square, or error variance, estimate (this estimate 
would be used later in comparing predicted values to observed values). The statistical 
model for the response as a function of the contributing factor settings was then 


^(0j, MIBj, K^) = JI + (\Xq. - ll) + CM-MIBj - W + (^K xk - H) 

where )i(0j, MIBj, K Xk ) is the projected mean response with glide path angle equal 0j, 
minimum impulse bit mode set to MIBj and the approach gain equal to K Xk ; 

ji is the overall mean of all observed responses; 

fig. is the estimated effect of 0 at setting i; 

M-MIBj ' s the estimated effect of MIB at setting j; 

|iK xk is the estimated effect of K x at setting K x ^. 

The next step was to compare predicted values to actual observations from the simulation 
program at various combinations of settings for the contributing factors. Predicted values 
for the mean response at given factor settings are obtained from the functional description 
above. Observed means would be estimated (via sample mean values) from several runs 
made at the corresponding settings through the simulation program, where random initial 
state errors were included to approximate actual "noise" effects (the nine data points 
obtained through the experimental design were outcomes resulting from no initial state 
errors). 

Table 2 on the following page lists the model predictions for total RCS at all 18 
combinations of the significant design factor levels. Predictions are arranged in ascending 
order to facilitate a rank-order comparison with observed values in the adjacent column. 
Observed sample means in the third column are for samples of size 12 at selected level 
combinations (time did not permit simulation and review of data at all combinations). The 
standard deviation used in computing the normalized error is a function of the ANOVA 
estimated error variance and the dispersion observed in the 12 replications at that 
particular combination of factor levels. We note that the standardized errors are all of 
reasonable magnitude, except for that at the predicted minimum (factor combination #1), 
and the rank-order of predicted RCS values agrees closely with that for the observations. 

Figure 2, which follows the table, depicts a graphical comparison of the data. The actual 
range of observed values is about 64% of the range predicted, with the most significant 
error, again, being that at the optimum combination. However, we note that with this one 
exception all predictions fall within a 2-sigma error envelope (shown as the area between 
broken lines in the figure) used as a rule-of-thumb upper bound by some authors. Thus, 
while the model predictions are not uniformly in close agreement with the corresponding 


4-6 



Total RCS 


TABLE 2: COMPARISON OF MODEL PREDICTIONS WITH SIMULATION OUTPUT 


Factor Combination 

Mean - Model 

Mean - Simulation 

Error 

Error 

Count 

Prediction 

Observations 

(Percent! i 

in Std. Dev.'s! 

1 

.838 

.962 

-12.88 

4.00 

2 

.895 




3 

.940 

.997 

- 5.68 

1.14 

4 

.990 




5 

1.005 




6 

1.090 

1.112 

- 1.96 

0.48 

7 

1.111 




8 

1.165 

1.180 

- 1.26 

0.33 

9 

1.224 

1.120 

9.29 

1.93 

1 0 

1.230 




1 1 

1.288 




1 2 

1.307 

1.252 

4.42 

0.94 

1 3 

1.314 

1.347 

- 2.45 

0.61 

1 4 

1.381 




1 5 

1.445 




1 6 

1.453 




1 7 

1.475 

1.369 

7.78 

1.66 

1 8 

1.631 






Factor Combinations 


prediction 

actual 


Figure 2: Total RCS Predicted Means vs. Observed Means 
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observations — particularly, at the predicted extremes — the model seems to capture 
adequately information regarding rank ordering of the possible factor combinations. 


CONCLUSIONS 


Our assessment is that the primary source of differences between the projections of the 
statistical model and observations obtained through simulation is most likely a single 
interaction effect between two of the factors. Recall that the Lg experimental design 
employed provided for estimation of no factor interactions. Thus, any interaction effects 
that are, in fact, present are confounded with some subset of the main effects. However, 
given the level of agreement between predictions and observations over the range of 
results, we feel that the postulated interaction effect, though not insignificant, may be a 
mild one. We emphasize that provision for evaluation of the interaction would likely lead 
to a mathematical model with improved predictive capabilities; however, our chief 
objective has been to determine a near-optimum combination of design factor settings, and 
we feel that we have been able to do that. 

For this investigator and the NASA host group, this has been a first exercise with Taguchi 
methods in a realistic problem setting. Our overall evaluation of the technique is that it 
can prove to be an efficient analytical tool, particularly once one has gained some 
experience with the mechanics and develops a feel for its subtleties. As a summary, we 
express impressions formed and offer some caveats in working with Taguchi analysis. 

Rule-of thumb rationale for identifying "significant" factors will not always produce a 
clear-cut set of contributing variables. In typical ANOVA, a level of significance for 
statistical tests may be specified, and the F-ratio suggests that either a factor is 
significant or it is not. This is not the case here, it may be extremely difficult in this 
application to specify an alpha value in advance and then adhere to the conclusions one is 
drawn to as a result. A Pareto Principle approach seems more practical. This maxim 
suggests that approximately half the factors (actually, the factors associated with 
approximately half the total degrees of freedom) should account for the large majority of 
the observed variation. These factors are significant, and the sum of squares attributed to 
the remainder of the variables (i.e., the nonsignificant factors) are pooled with the error 
variance. This rationale is likely to work well in most instances — but not always. There 
will be "borderline" cases where it is difficult to feel confident about assigning a factor to 
either class. We can only suggest that the best aid in such situations is one's insight into 
the phenomena under study. 

Computation of squared deviations from a target follows from Taguchi's paradigm of the 
quadratic quality loss function. For cases where deviations from the target are more easily 
accommodated in one direction than the other, authors in the field offer the useful concept 
of employing asymmetric penalties wherein computations produce scaled deviations of 
k^A 2 in one direction and k 2 A 2 in the other. We feel that in many engineering design 
problems such asymmetric penalties may be called for. 
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We have found through our experience with this study that there may be some difficulty, 
when using small orthogonal arrays with all columns assigned to factors, in determining 
whether significant interactions are present and in deciding with some confidence which, 
if any, these might be. The graphical techniques suggested as offering some indication of 
the presence of interactions are based on comparison of mean effect against mean effect at 
various levels for two factors. In small experimental designs, these mean effects may be 
based on a single observation so that the standard error of the mean is relatively large. 
Evaluation is thus clouded by these uncertainties. A practical method may be to compare 
predictions against actual observations at various points in the domain as we have done 
here; lack of close agreement may suggest interaction effects not provided for in the 
experimental design. 

Despite the possibility of overlooking interactions with small, total column-assigned 
array designs, much information can be gained if we view analysis at this stage as a first 
step in an iterative process. Frequently, it will be desirable, after performing a 
preliminary analysis, to "fine tune" the spacing of settings for the significant factors in an 
attempt to incrementally improve upon the optimum initially identified; thus, a second 
iteration may be planned as a matter of course. If we conclude that there may exist notable 
interactions and can determine which these might be, design of the experiment array for 
the subsequent stage can take this possibility into account. 

In evaluating supporting software, we would look for special applications that can assist in 
selection and modification of the standard orthogonal arrays, advising the analyst as to 
which experimental design might be most appropriate for the problem at hand. Any 
software package that performs only the necessary statistical calculations may offer little 
more than existing nonspecific statistical packages. 

Engineering design problems, particularly those undertaken within NASA, typically 
involve concern with multiple objectives. It may be possible to scale and weight the 
various individual objectives and pool these so as to enter into the Taguchi analysis with a 
single (weighted-sum) objective. However, it is not clear whether we can expect results 
with a similar level of integrity as those deriving from analyses with single-criterion 
objective functions. It may be necessary to carry out evaluations on a piecemeal basis 
(i.e., through independent consideration of the various objectives) and then perform a 
constrained trade-off analysis to ascertain near-optimal solutions. 

We should underline the fact that the philosophy of Taguchi design optimization is to 
produce a quality design through a structured experimentation process which yields a 
maximum amount of information about the system under study with a minimum 
investment of time, effort and cost. As with many other methods within the TQM field, 
realization of increased efficiency depends on a shift of major effort to the early steps in 
the project. In Taguchi analysis it is engineering experience and intuition that are called 
upon at the beginning of the process in postulating controllable design factors and potential 
interactions and in establishing a feasible region of parameter settings to produce a 
desired result. Failure to take advantage of these elements can reduce the return promised 
by the methodology. 


4-9 



BIBLIOGRAPHY 


[ 1 j Hicks, C.R.; Fundamental Concepts in the Design of Experiments ; CBS College 
Publishing, 383 Madison Avenue, New York, N.Y. 10017; 1982 

[2] Phadke, M.S.; Quality Engineering Using Robust Design ; Prentice Hall, Englewood 
Cliffs, N.J. 07632; 1989 

[3] Roy, R.K.; A Primer on the Taguchi Method ; Van Nostrand Reinhold, 115 Fifth 
Avenue, New York, N.Y. 10003; 1990 


4-10 



